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ABSTRACT 

Objectives: To describe experiences with the 
implementation of global trigger tool (GTT) reviews in 
five Danish hospitals and to suggest ways to improve 
the performance of GTT review teams. 
Design: Retrospective observational study. 
Setting: The measurement and monitoring of harms 
are crucial to campaigns to improve the safety of 
patients. Increasingly, teams use the GTT to review 
patient records and measure harms in English and non- 
English-speaking countries. Meanwhile, it is not clear as 
to how the method performs in such diverse settings. 
Participants: Review teams from five Danish pilot 
hospitals of the national Danish Safer Hospital 
Programme. 

Primary and secondary outcome measures: We 

collected harm rates, background and anecdotal 
information and reported patient safety incidents (PSIs) 
from five pilot hospitals currently participating in the 
Danish Safer Hospital Programme. Experienced 
reviewers categorised harms by type. We plotted harm 
rates as run-charts and applied rules for the detection of 
patterns of non-random variation. 
Results: The hospitals differed in size but had similar 
patient populations and activity. PSIs varied between 3 
and 12 per 1000 patient-days. The average harm rate for 
all hospitals was 60 per 1000 patient-days ranging from 
34 to 84. The percentage of harmed patients was 25 
and ranged from 18 to 33. Overall, 96% of harms were 
temporary. Infections, pressure ulcers procedure-related 
and gastrointestinal problems were common. Teams 
reported differences in training and review procedures 
such as the role of the secondary reviewer. 
Conclusions: We found substantial variation in harm 
rates. Differences in training, review procedures and 
documentation in patient records probably contributed 
to these variations. Training reviewers as teams, 
specifying the roles of the different reviewers, training 
records and a database for findings of reviews may 
improve the application of the GTT. 



Correspondence to 

Dr Christian von Plessen; 
cple@hih.regionh.dk 



BACKGROUND 

Patients run a high risk of being harmed 
during hospital admissions. Adverse events 
occur in up to 10% of hospitalisations and 



I 



ARTICLE SUMMARY 



Article focus 

■ To describe experiences with the implementation 
of global trigger tool (GTT) reviews in five 
Danish hospitals. 

■ To suggest ways to improve the performance of 
GTT review teams. ^ 

Key messages 

■ Differences in training, review processes and 
documentation can contribute to differences GTT 
results. 

■ Systematic training of review teams, standardisa- 
tion of review processes and feedback on results 
should improve the performance of review teams^ 

Strengths and limitations of this study 

■ The study supplies detailed contextual informa- 
tion that is relevant for the implementation of the 
GTT that increasingly is being used to monitor 
the safety in hospitals. 

■ The observational study design dose not allow 
for conclusions on causality and quantification of 
the contributions of different factors with rele- 
vance for the Implementation of GTT reviews. 



can cause death, permanent or temporary 
disability.^ For patients and healthcare 
workers, these harms and the underlying 
flaws of their healthcare systems that permit 
them to happen are deeply upsetting and 
completely unacceptable. 

To improve the safety of patients, national 
and regional campaigns have been carried 
through^"'^ or are ongoing.'' Improvements 
have been achieved in some areas such as 
reductions of catheter-related blood stream 
infections. However, system-wide progress is 
slow*^ and improvements are often limited to 
particular medical conditions or institutions. 
Indeed, a recent study^ from the state of 
North Carolina, an active participant in 
large-scale patient safety initiatives, concluded 
that overall rates of harm during 2002-2007 
were not reduced. Thus, the challenge to 
improve the safety of patients in hospitals 
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remains and specific and sensitive mejisures of harms are 
needed to assess and monitor the effects of changes to 
make hospitals safer. 

In Denmark, the Operation Life campaign during 
2006-2008 focused on patient Sctfety in intensive care and 
during surgery. An estimated 1654 fewer than expected 
patients died in the Danish population of 5.5 million 
during the campaign.^" In 2010, another campaign the 
Safer Hospital Programme (http://www.sikkerpatient. 
dk/fagfolk/patientsikkert-sygehus.aspx) was launched at 
five pilot hospitals to reduce mortality by 15% and harms 
by 30% through the implementation of 12 care bundles. 
The hospitals are required to measure and report harms. 

Meanwhile, a gold standard for the measurement of 
harms does not exist. Methods like voluntary reports only 
detect a small fraction of harms, chart reviews have low 
inter-rater reliability^^ and are very time consuming and 
so are direct observations of healthcare processes.^'' 
Studies comparing different methods of harm detection 
have found very littie overlap of the detected harms. 
Therefore, complete estimates of the incidence of harms 
probably require the combination of different methods. 
Meanwhile, such an approach is time consuming and 
results are often delayed, which is unsuitable for patient 
safety campaigns in which frequent and regular measure- 
ments of harms are needed to evaluate and monitor the 
effects of interventions and organisational changes. 

The global trigger tool (GTT) has been developed for 
the purpose of monitoring harms at low cost.^'^ Harm in 
this context is defined as an 'Unintended physical injury 
resulting from or contributed to by medical care that 
requires additional monitoring, treatment or hospitalisa- 
tion, or that results in death'. Thus, the tool measures 
factual harm to patients, while errors not leading to 
harm, near errors and errors of omission are not 
included. A GTT review is a trigger-based chart audit of 
closed patient charts. Two reviewers, usually nurses or 
pharmacists, each review a limited number of randomly 
chosen charts with a given set of 56 triggers or "hints" of 
errors. The finding of such a hint triggers an investiga- 
tion into whether and, if so, how severely, a patient actu- 
ally has been harmed. Review time is limited to 20 min 
per admission. Finally, the two reviewers compare their 
conclusions and a supervisor, usually a physician, quali- 
fies the number and severity of harms and decides in 
cases of disagreement. The number of harms is then 
expressed as a rate, for example, harms per 1000 
bed-days. It has been suggested that GTT teams need a 
limited amount of training and practice to achieve good 
levels of reliability to identify harms. ^'^^^^ The feasibility 
of the method invites for rapid adoption in healthcare 
systems around the world where practical ways to 
measure harms are much in demand. Nevertheless, 
experiences with the GTT in non-English-speaking 
countries are limited. Thus, careful calibration of the 
instrument and the review team that uses it is warranted 
to avoid evaluating the satfety performance of hospitals 
with imprecise measurements. 



A team of Danish experts translated the GTT to 
Danish^^ from the English and a Swedish version.^" The 
tool was tested in four hospitals in different health 
regions. The harm rate in these hospitals was around 
20 per 1000 bed-days. A recent report of harms to Danish 
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Figure 1 Harms by type. The dot plots show the relative 
distribution of harms by type. VIE, venous thromboembolism. 
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patients with cancer found a rate of 68 per 1000 
bed-days. Notwithstanding these variable rates, policy 
makers advocate the widespread^'^ implementation of 
GTT reviews in Danish hospitals. Meanwhile in our 
opinion, it is not sufficientiy clear as to how the tool per- 
forms in the hands of Danish review teams. 

The aim of this study was to describe experiences with 
the GTT in five Danish hospitals and suggest ways to 
improve the performance of GTT review teams and thus 
contribute to the accurate measurements of harms. 



METHODS 

In this retrospective observational study, we present harm 
rates as measured by the GTT at the five hospitals partici- 
pating in the Safer Hospital Programme in Denmark. 
The hospitals had used the GTT for 18 months from 
January 2010 until June 2011. The project managers at 
the hospitals supplied tabulated data on the size, activity 
and patient populations of the five hospitals from the 
year 2010. All data were collected between August and 
December 2011. 

CvP, the project manager at one of the five hospitals 
and a consultant in pulmonary medicine, and AMK, a 
registered nurse and experienced GTT reviewer, inter- 
viewed members of the GTT teams and project man- 
agers on the telephone. The interviews comprised 
questions on the training of the GTT teams, team com- 
position, roles of the team members and review pro- 
cesses. Moreover, they asked open questions about 
unexpected observations and changes. The GTT teams 
supplied lists of the recorded harms from their reviews. 

To give an impression of the safety culture at the par- 
ticipating hospitals, we present the number and severity 
of reported patient safety incidents (PSIs) in 2010. JA, a 
physician by training who works in the Danish Society 
for Patient Safety, gathered these data from the Danish 
Patient Saifety Database (www.dpsd.dk) for voluntary 
reporting of PSIs. Risk managers at the five hospitals 
classified the severity of PSIs in the Danish Safety 
Database into mild, moderate, severe and fatal. In 
Denmark, reporting of PSIs is mandatory, confidential 
and sanction free for healthcare personnel. 



GTT REVIEW 

The Danish translation of the GTT toolkit was the refer- 
ence for the review teams. ^'"^ Teams were to review a 
random sample of 10 admissions twice a month. Closed 
admissions of patients of at least 18 years of age and of 
at least 24 h duration were eligible. The date of dis- 
charge was the index date. The GTT teams should 
review all available information from the admissions, 
that is, physician and nurse notes, medication orders 
and history as well as results of diagnostic tests. Each 
primary reviewer should review each record. Afterwards, 
the primary reviewers should meet to discuss harms and 
come to consensus. Finally, the primary reviewers should 
meet with the supervising physician to present their 
findings for approval and severity classification. Triggers, 
harms and the severity of harms should be recorded on 
standardised work sheets, then transferred to spread- 
sheets and stored locally. The team should classify the 
severity of hjirms according to the National Council for 
Medication Error Reporting and Prevention Index. 
The five hospitals only register harm rates in a shared 
database other data from GTT reviews are stored locally. 



DATA ANALYSIS 

We calculated the monthly harms per 1000 bed days, 
harms per 100 admissions and the percentage of harmed 
patients. The harm rate was then plotted on a run chart 
and analysed using four run chart rules for the detection 
of patterns of non-random variation such as shifts or 
trends. ^""^ We collected and managed the data in 
Microsoft Excel V.2003 and produced the statistical ana- 
lysis and graphs in R Statistical Software V.2.13.1. 

For the purpose of this study, two nurses from the 
GTT team of one of the hospitals who had done over 
400 GTT reviews each retrospectively categorised harms 
found at all five hospitals into categories also used by 
Classen et al}^ They added gastrointestinal complica- 
tions and pressure ulcers as categories because these 
were common types of harm (figure 1). Each harm was 
assigned to only one category. 

The regional ethical committee deemed an ethical 
review of the study unnecessary. 



p ^ 

Table 1 Background information on tfie five fiospitals (201 0) 




Hilleroed 


Horsens 


Kolding 


Naestved 


Thy-Mors 


Discfiarges 


60 098 


30 377 


27 526 


28 677 


11 836 


Average patient age (years) 


55 


57 


53 


59 


58 


Percent of females 


62 


59 


61 


55 


55 


Patient-days 


231 978 


108 060 


90 710 


113 353 


49 711 


Outpatient visits 


262 547 


212 899 


124 184 


184 374 


65 165 


Employees 


3163 


1367 


1507 


1668 


689 


Hospital standardised mortality rate 


95 


97 


96 


112 


100 


Reported safety incidents 


2736 


365 


923 


1182 


223 


Reported patient safety incidents per 1 000 patient-days 


12 


3 


10 


10 


4 
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RESULTS 

Background data of the five pilot hospitals 

The five hospitals, one in each Danish health region, 
vary in size but all hospitals have departments of internal 
medicine, orthopaedic and general surgery as well as 
obstetrics and gynaecology. All hospitals use electronic 
patient records, but to varying degree parts of the docu- 
mentation, such as nursing notes, are on paper. The 
populations of patients at the five hospitals were similar 
with regard to age and gender distribution. There was a 
fourfold difference in reporting of PSIs among hospitals 
(table 1). Meanwhile, the distributions of PSIs by conse- 
quence were almost identical ( figure 2) . 

Experiences with the implementation of the GTT 

Reviewing patient files with the GTT was new to three of 
the five teams. Prior to the Safer Hospital Programme, 
Naestved and Hilleroed Hospital had been using the 
GTT for reviewing patient records for 1 and 2 years, 
respectively. 

In May 2010, the GTT team from each of the five hos- 
pitals participated in a 7 h training session with experts 
in the method from Denmark and the Institute of 
Health Care Improvement (IHI). The session included 
an introduction to the Danish GTT manual, frontal 
teaching, review of three training records per team and 
plenary discussions of the findings (table 2). Only the 
review team at Hilleroed had in 2008 received a similar 
training. All teams used nurses as primary reviewers and 
physicians as supervisors. All teams received on-site 
expert coaching with reviews of 10 or more records, up 
to three times. The expert coach was a physician who 
was trained by experts from the IHI. Furthermore, all 
teams participated in two full-day network seminars 
during the study period. All teams started reviewing 
patient records for the measurement of their baseline in 
May 2010 and retrospectively reviewed records from 
January to May 2010. 

The compositions of teams changed between zero 
(HUleroed and Naestved) and three times (Thy-Mors) 
during the study period. Review intervals at the hospitals 
varied between 2 weeks, 1 month and irregular. 
Complete teams reviewed together and compared find- 
ings at two, later three hospitals. The role of the phys- 
ician in the review team varied from only judging cases 
where the primary reviewers were in doubt or disagreed 
(Hilleroed) to identifying harms based on triggers 
found by primary reviewers (Horsens). Table 2 shows dif- 
ferences in review procedures at the hospitals. 

Anecdotal information about GTT reviews 

At Naestved, the team sampled 24 records (in case some 
were incomplete) each month and sorted them in the 
order of the date of the admission and reviewed the first 
20 records. Thus, the sample became biased towards 
admissions in the earlier part of the month. Moreover, 
the tCcim initially reviewed only admissions to the last 
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Figure 2 Patient safety incidents by consequence. The dot 
plots show the relative distribution of patient safety incidents 
by consequence as reported to the Danish national database. 
Categories minor and moderate represent no and temporary 
harms, ma/or permanent harms. Overall, 96% of the incidents 
were temporary. 



4 



von Plessen C, Kodal AM, Anh0j J. BMJ Open 2012;2:e001324. doi:10.1136/bmjopen-2012-001324 



Experiences with the global trigger tool in a non-English-speaking country 



Table 2 Characteristics and review procedures of global trigger tool teams at five hospitals ^^^B 




Hilleroed 


Horsens 


Kolding 


Naestved 


Thy-Mors 


Team characteristics 










i 


Number of physicians 


o 


1 






J 


Number of nurses 


o 

o 


o 
o 


4 later o 


4 


3 V 


Number of changes in team 


U 


1 

(physician) 


1 


r\ 
U 


d. 

(nurses) 


Review intervals 


Twice per 
month 


Monthly 


Monthly (two 
half days) 


Monthly 


Variable 


Training 










, 1 


Hours of training 


14* 


7 


7 


7 


i 


Site visits by Danish expert (days) 


1 


o 
o 


1 


4 




Complete team present during site visit 






+ 




1 


Number of records reviewed together with expert 


<10 




>\0 


>10 


>10 1 


Review procedures 










1 


vvnoie leam meeis Tor reviews 


+ 


+ 


oince Jan. 
2011 






Physician acts as judge (J) in cases of 


J 


R 


J 


J 




disagreement or reviewer (R) based on triggers 












Records entirely electronic 




+ 


+ 






Dedicated person responsible to find records 


+ 




+ 


+ 




Secretary plots triggers and harms 










+ 



*Team also trained by national expert in 2008. 



department of a hospital admission. Thus, they did not 
find harms that, for example, occurred during an admis- 
sion to the intensive care unit earlier during the hospital 
stay. These errors were accidentally discovered during a 
site visit and the team did a new review for the period. 
The team at Kolding discovered atfter 3 months that 
their sampling procedure excluded admissions that had 
an appointment for ambulatory follow-up after surgery, 
and they decided to discard the first 3 months from 
their baseline. 



GTT findings 

In total, 687 adverse events were identified in 11 487 
patient-days, that is, the overall average harm rate was 60 
per 1000 patient-days. The monthly harm rate ranged 
from 34 to 84 harms per 1000 patient-days (figure 3). 
The harm rates at all five hospitals showed only random 
variation. The overall numbers of harms per 100 
admissions were at Thy Mors 45, Naestved 24, Kolding 
30, Horsens 43 and Hilleroed 54. The percentage of 
harmed patients was 25, ranging from 18 (Horsens) to 
33 (Hilleroed) (figure 4). 

Overall, 96% of harms were temporary (grades E and F). 
However, the severity distribution varied between hospi- 
tals (figure 5). Notably, the hospital with the highest 
harm rate (Hilleroed) also had the highest proportion 
of grade E harms. 

Five hundred and fifty-three harms (80%) were 
recorded with sufficient detail in the hospital datasheets to 
categorise them by type. The proportion of harms without 
description varied: 5% (Kolding and Hilleroed), 12% 
(Thy-Mors and Horsens) and 45% (Naestved). Common 



types of harm were infections, procedure-related, pressure 
ulcers and gastrointestinal problems (figure 1). 



DISCUSSION 

We observed marked differences in the harm rates and 
types identified by GTT review teams in five Danish 
public hospitals. The GTT is not designed to compare 
hospitals but we were surprised by the magnitude of the 
variations. Therefore, we designed this study to identify 
factors that contributed to the differences. The hospi- 
tals, their patient populations, structures and activity 
levels were similar but we found differences in the train- 
ing, the review procedures and the experience of the 
review teams. 

Other studies have also found variation of harms 
across hospitals. Naessens et al^^ in a study of 1138 
admissions to three academic health centres in three 
states of the USA found a variation of harms by hospital 
between 23.1% and 37.9% of admissions. In a study of 
surgical harms, the variation was between 5% and 
35%, Sharek et aP observed harm rates between 
0.18 and 1.28 per patient in 749 admissions to 15 
newborn intensive care units in the USA and Canada 
and Resar et aiP in 62 intensive care units in the USA 
measured between 3.2 and 27.36 harms per 100 days. 
Thus, significant variations of GTT findings seem to be 
common. 

Several factors could explain the variation in rates of 
harm. First, there can be differences in the Sctfety of 
the clinical processes at the hospitals. However, it seems 
unlikely that such differences should cause as much as 
a 2.5-fold variation of harm rates given the similar 
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Figure 3 Rates of harm. The run charts show monthly rates 
of harms measured with the global trigger tool. The curve 
shows the harm rate expressed as the number of adverse 
events per 1000 patient-days. The horizontal line is the 
median harm rate. 
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Figure 4 Harmed patients. The run charts show the 
percentages of harmed patients measured with the global 
trigger tool. The horizontal line is the median percentage of 
harms. 



patient populations at the five hospitals and the homo- 
geneity of the Danish healthcare system in general. We 
even found that the hospital with the highest mean age 
and the highest hospital standardised mortality rate had 
the lowest rate of harms. Second, the documentation of 
triggers and harms probably varies across hospitals. 
High PSI reporting rates are generally considered a 
sign of a mature safety culture rather than of poor 
safety and one could speculate that staff is more likely 
to document harms in a hospital with such a culture. 
Also different types of patient records (electronic and 
paper) and differences in layout and presentation 
could influence the results of the reviews. We do not 
have data to explore these questions but we certainly 
cannot exclude an influence on the different harm 



rates across hospitals. Third, differences in the training 
and the experience of the review teams influence the 
subjective process of judging harms in any record 
review.^^ The team that found the most harms was the 
most experienced review team and had attended two 
training seminars. This team was also stable and 
reviewed regularly with the whole team present twice a 
month. Finally, the teams conducted the review pro- 
cesses in slighdy different ways. Most importandy, the 
roles of nurses and physicians varied. The role of the 
physician in the review team that found the highest 
harm rate was to judge in cases of disagreement while 
physicians in the other teams themselves identified 
harms. We assume that nurses are more prone to regis- 
ter harms of lower severity, while physicians might 
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Figure 5 Harms by severity. The dot plots sliow tlie relative 
distribution of severity of harms in categories E — I, where E 
and F are temporary, G — H permanent harms and / death. 
Overall, 96% of harms were temporary. 

consider them insignificant. This interpretation is sup- 
ported by our finding that the variation of harm rates 
was greatest in the least severe category 'E'. 

Thus, in our opinion, the experience of the GTT teams 
and the way they perform the reviews strongly contribu- 
ted to the differences in harm rates in the five hospitals. 



Moreover, differences in the documentation of harms in 
the patient records seem to influence the number of 
harms the GTT team can find. We did not expect these 
factors to be so important because the GTT was imple- 
mented according to current recommendations^'^ ^' and 
was guided by some of their authors. Moreover, all the 
teams had attended a GTT network meeting with 
national and international experts and received site visits 
by a national expert. However, these precautions, it 
seems, were not sufficient to secure at standardised 
reviewing process at the hospitals. Thus, users of the GTT 
and its results, healthcare personnel, administrators, 
payers or the public, should be aware of the challenges of 
the implementation of the method and allow for suffi- 
cient training and evaluation of the results. Our findings 
also stress that GTT findings should guide hospitals in 
their efforts to improve patient satfety bvit the results 
should not be used to compare hospitals. 



STRENGTHS, LIMITATIONS AND FURTHER RESEARCH 

The strength of this study is its relevance for the imple- 
mentation of the GTT that increasingly is being used to 
monitor the satfety in hospitals. Our contextual data are 
detailed and thus practical. The limitations are inherent 
to the observational nature of the study that prevents 
conclusions on causal links between the variations of 
harm rates and the observed differences in team train- 
ing and review processes. For the same reason, we 
cannot quantify the contribution of the different factors. 
Nevertheless, the findings are plausible and fit with the 
recommendations for the use of the method. ^'^ 

Further research should address how teams' reviewing 
experience and training influence team performance 
and how team training can be optimised. Moreover, 
studies should investigate the influence of changes of 
documentation and presentation of information in 
patient charts and the use of the types of harm for 
improving patient safety. 

In conclusion, differences in training, review processes 
and documentation probably have contributed to varia- 
tions in rates of harm as measured by the GTT. Thus, 
healthcare staff and policy makers should be aware of 
the need for systematic training of the review teams and 
standardisation of the review process when implement- 
ing the GTT in new settings. These factors are related to 
the implementation of the GTT reviews and are not 
inherent to the method as such. Our study has implica- 
tions for the implementation of the method in other 
settings and we suggest considering the following inter- 
ventions to improve the implementation of the GTT in 
new settings: 

► Secure that the review team is trained as a team. 

► Specify the roles of the reviewers during the reviews 
to avoid overestimation/underestimation of especially 
harms of lesser severity depending on professional 
background. 
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► Test review teams' abilities to find harms with a set of 
training charts to estimate their 'sensitivity' before 
routine monitoring is instituted. 

► Define a minimum number of patient charts that the 
team should have reviewed before monitoring harms 
routinely. 

► Perform reviews with all team members present. 

► Ensure a structured review process, that is, a space 
where the team can work without interruptions, regular 
time intervals between reviews to keep team 'in shape'. 

► Implement a common databcise with individual 
patient data to allow for re-examination of reviewed 
charts to avoid problems such as sampling errors. 
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